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ABSTRACT 


PEeerimiGdt test ncadsin which the results of a simulation 
experiment can be reported and analyzed is described. The 
test bed is based on the regression adjusted graphics and 
estimation (RAGE) methodology developed by Heidelberger and 
Lewis {Ref. 1] for regenerative simulations. From the graphics 
and the associated numerics the experimenter can summarize 
and see simultaneously relative properties, such as bias, 
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I. INTRODUCTION 


During the course of a simulation experiment the experi- 
menter is frequently confronted with the problem of estimating 
some function or parameter from the generated data. He will 
usually have a choice of one or more estimators, parametric 
or nonparametric, that can be used to estimate the function 
Or parameter. These estimators will vary in their statistical 
Peoperties. some will estimate with low variance but contain 
bias, others will have just the opposite characteristics and 
Still others will contain varying degrees of both. How these 
different estimators compare in relation to each other as 
well as over varying sample sizes are of concern to the experi- 
menter. Comparison of estimators are important in that they 
help the experimenter to choose the most appropriate estimator 
aomemils particular analysis. Comparisons may also be helpful 
fimecaeciding whether a new estimator is of any use. The appro- 
priate estimator to use is a function of the bias, variance 
and nonnormality in the estimate for available run length 
(sample size). 

When the simulated data from which the properties of the 
estimators is to be examined is independent and identically 
distributed (iid) it can be sectioned into M independent 
blocks of specified sample size (N). From each of these 


blocks the estimator can be applied to all subsamples of 


aH 





Size Nh» thereby creating samples of estimates of varying 

aetiemeor Cacil estimator. As a £irst cut at the properties 

of the estimators, box plots of these samples of estimates 

are drawn for subsample sizes a (see, e.g. Figure 3a). 
However, something is usually known about the properties 

of the estimators and these properties can be used to enhance 

the graph of box plots versus subsample size, ny. Thus it 

is often known that bias in the estimator is a polynomial 

in l/n, and the variance of the estimator is a polynomial in 


i, I , I=, ... Also the estimators are usually asymp- 


totically normally distributed. Consequently the graphs are 
enhanced by an estimated unbiased regression line for the 
mean of the estimates of the parameter as well as a horizontal 
line for the regression-estimated expected value of the 
parameter (asymptote). A regression is also performed for 
the variance structure of the estimators and the sample 
characteristics (kurtosis and skewness) are given under each 
box plot to assist in judging the convergence to normality. 
Heidelberger and Lewis [Ref. 1] used this methodology in 
the context of regenerative simulation to develop a procedure 
and protocol to produce unbiased and normally distributed 
regression adjusted regenerative estimates (RAGE), with a 
given precision or for total available computing time. How- 
emer, the graphics associated with this procedure required a 


Tektronic 4015 terminal using an APL graphics package making 
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gis Use limited to those organizations possessing such a 
terminal. The protocol also requires a relatively experi- 
enced user to properly interpret the graphical and statisti- 
cal output and to apply the sequential steps of the protocol. 

The purpose of this thesis is to apply the methodology 
developed by Heidelberger and Lewis [Ref. 1] to comparing 
the bias and variance of estimators of a parameter from simu- 
lated output. This objective also includes supporting the 
graphics associated with the methodology on a line printer 
to make its use more accessible. 

Details of the graphics and methodology are given in 
Siepeer If, entitled USAGE. Details of the arguments of the 
RAGE program are given in Chapter III and a sample program 
Msing the RAGE graphics is given in Chapter IV. All the 
user has to supply is the input data and FORTRAN functions 
to compute the estimators for any given subset of data. 

A comparative study of estimators of the gamma shape 
parameter using the graphical test bed RAGE is given in 
M@imapter V. This comparative study illustrates the different 
properties of estimators that can be seen with regression 


adjusted graphics as sample size increases. 


Ls 





i ieeWonGi 


The regression adjusted graphics and estimating (RAGE) 
program is designed to evaluate distributional properties of 
SiewOn more estimators as a function of sample size from a 
Stationary series of raw data. These estimators are program- 
med as functions by the user and declared external in the 
main program. The graphical and numerical results that 
accompany each estimator are varied and will depend to some 
extent on the options desired by the user. The purpose of 
the following discussion is to explain the graphical and 
numerical output as well as how they were obtained and some 
of the options available to the user. 

The general procedure begins with a given iid data base 
or one that has been simulated and an unknown parameter or 
population characteristic, 9, that is to be estimated by an 
estimator, 6, from the data. The experimenter must first 
decide on the number of independent blocks or sections (M) 
of size N the data is to be sectioned into. These M sections 
or blocks will henceforth be referred to as replications; 
the terminology will seem more appropriate as the discussion 
continues. The number of replications (M) can alternatively 
be determined indirectly by the sample size (N) desired in 
each section. In either case MxN must not exceed the total 


allowable sample size. This sectioning of the data into 
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Peeeplications Of sample size N is represented in Figure 1] 
where the entire block represents the MxN samples from the 


total sample population. 


Pigure 1: M replications, each of sample size N 

Within each replication the experimenter must decide on 
the subsample sizes that the estimator, 6, is to use in 
estimating the parameter. The number of different subsample 
sizes, K, that can be used to resection the same data can vary 
from one to eight and the size of each subsample can vary 
from one to the size of the sample, N. This repeated sub- 
sectioning of the same data is represented pictorially by 
Figure 2 where [x] refers to the highest integer less than 
Meedial to x and r;,, k=1,..., K refers to the number of 
estimates possible from a sample of size N using subsample 
Sizes Ny, k=1,..., Kto estimate the parameter. Each box 
Or rectangle in Figure 2 represents the same MxN data set 
with the same M replications of size N subsectioned into ° 
Geererent sections of size n,. Note that the n,'s divide 
Smevenly in Figure 2. This is not necessary but certainly 


feeerapble since all available information will be utilized. 


des 





E@mulation oO: 











Sample section Estimates for 
size Total Samples MxN OSD) 5 Seusanioke Sze Tl 
N et 
N Hy) 
N {x M = r4M 
aol 
N my, 
N My] 
N m7 
Nix M = T5M 
oy: 
N my 
N m4 
N m4 
N {|x M = TM 
a 
— 
N 1 
Figure 2: Subsectioning MxN samples into subsamples 
of size ee LS ceca |S 
Pameden subsample Size (m,, k=l, ..., K) produces a total 
Or Ty, k=1,.-.-., K estimates per sample of size N then there 


Will be a population of estimates from the M replications 


mee tOtal Mxr;, k=l,.. K for each subsample. For each 


2-3 


16 





wepmtation of estimates From each subsample (Ny >» oreo: 
Pexepiloats are plotted so that the distribution of the esti- 
mates for the varying subsample sizes can be seen graphically. 

In addition to these box plots of the estimates, a 
regression adjusted estimate (RARE) of the parameter is 
memputed and plotted over the boxplots. In addition to the 
RARE estimate, unbiased RARE estimates out to a specified 
number of terms determined by the user are computed and plotted 
on the graph (The mathematical derivation of the RARE and 
unbiased RARE estimates are given later in this section). 

The unbiased RARE estimates are computed as a function of 
subsample size and plotted in conjunction with the RARE 
estimate so that a comparison of the two values can be made 
graphically and the evolution to an unbiased state can be 
feem as the subsample size increases. 

Pitcmiadea caiepe madesclearerethrough the use of an example 
and its associated graphics and numerics, Figure 4a. Here an 
independent and identically distributed sample of 50,000 
gamma distributed random numbers (shape parameter, K=5.0) 
has been generated and sectioned into M=100 sections (replica- 
tions) with each replication containing N=500 randon numbers 
Meample size). This information is printed at the top of 
the graph for future identification. The experimenter chose 
suene different subsample sizes at which to examine the 
Meeameter (335, 50, 71, 125, 166, 250 and 500). These 


subsample sizes (n,'s) are printed on the horizontal scale 


ys 





of the graph. The population of estimates of the parameter 
at each subsample size are then plotted as vertical box plots 
above these numbers. When the population of estimates deter- 
mined from a subsample size is less than nine, box plots 

ome omitted and the values of these estimates are simply 
plotted with the sumbol “O" on the graph. There are no 
estimates with populations totaling less than nine in Figure 
4a but the graphical output for such cases can be seen in 
Figure 3a where subsample sizes, 125, 166, 250 and 500 total 
fess than nine estimates. 

iai-mig@iZzentaleseale or the graph 15s scaled roughly to 
the minimum and maximum subsample sizes with the box plot of 
each subsample's estimates spaced proportionally. However, 
when subsample size differences are relatively small so that 
overlap of the box plots would occur, a minimum of six spaces 
Bemween Centers of box plots is left to prevent the graphics 
from becoming confused. 

The vertical scale of the graph is determined by the 
minimum and maximum values from the population of estimates 
determined from the smallest subsample size (n,)- It is 
assumed here that subsequent subsample sizes (n>, nz, ae ea) 
will be large in relation to the smallest subsample size 
(n,) so that estimates produced from these will be within 
the range of that determined from the smallest subsample 
Size. This assumption appears tenable if the range of esti- 


mates is compared in Figure 4a for increasing subsample sizes. 
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The vertical scale of Figure 4a is not scaled to the 
minimum and maximum value of the estimates determined from 
the smallest subsample size (n,=335). This is because the 
RAGE program can evaluate up to three different estimators 
at one time and two alternate estimators (Figures 4b § 4c) 
were also used to estimate the parameter. An option was 
entered to scale all three estimators to the same scale and 
it chose the minimum and maximum values from all the esti- 
mates produced by all three estimators using the smallest 
Suleample size (n,=55). By comparing Figures 4a, 4b and 4c 
it can be seen that the maximum value was taken from the 
first estimator, Figure 4a, and the minimum value was taken 
from the third estimator, Figure 4c. Other scaling options 
will be discussed in detail in the following chapter. 

Below the graph statistical results as they pertain to 
the population of estimates for each subsample size are 
printed. These statistics include the sample mean, standard 
deviation, standard deviation of the mean, skewness and kur- 
tosis. Thus, for example, in Figure 4a at subsample size 
iyo) there is a population of [N/n,]xM = 1 At |e (sou so) xl00)= 
150 estimates for which a sample mean, standard deviation, 
Standard deviation of the mean, skewness and kurtosis is 
computed. The subsample sizes these statistics refer to are 
labeled across the top. 

As mentioned earlier, in addition to box plots and associ- 


ated statistics of the estimates, RARE and unbiased RARE 


Ets, 





estimates of the parameter are computed and plotted on the 
@rapn. the degree of regression, D, for the computations of 
the unbiased RARE estimates is determined by the user and 
Meee ed at the top of the graph for future reference. The 
RARE estimate is the regression coefficient 8 9 and is the 
estimated asymptotic expected value of the parameter being 
estimated. The unbiased RARE estimates are determined by 
the regression coefficients, By, Bos -++s 8G, the subsample 
Size, n, and the RARE estimate, Bq» using the following 


mormula: 


1 +g, 1 


ie PL = J a + ed =a ‘ 








A set of regression coefficients (39; B < 84) is 


le 
determined from the mean of the population of estimates 
(dependent variable) at each subsample size for each replica- 


tion. The M regressions that determine the M sets of coeffi- 








cients (Bq; 84> Tie vemuepresenteda by the following 
mormula: 
1 IL 1 
me (My) Ses 84 “ + Bo et eee meal — 
i ny n 
fe 
mo (my) = 8g + 8) = + 8, _ eects + 8g oe 
k Ny MK 
Me St Tell 
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The regression coefficients, to determine the RARE and unbiased 
RARE estimates, are the average values determined from these 
M regressions applied to the M replications. The mean, 
Variance and standard deviation of these coefficients are 
printed below the graph below the subsample size statistics. 

PlcteCmemetEcdn Ter ommene fLairst coefficient, Bo, is the 
estimated asymptotic expected value of the parameter being 
estimated (RARE). Its mean from the M regressions is printed 
as a horizontal dashed line across the graph. As the sub- 
Sample size increases the bias in the unbiased RARE estimates 
becomes smaller and the unbiased RARE aecdmaneae (dotted line) 
will asymptotically approach the RARE estimate (dashed line). 
In Figure 4a the estimated expected value or asymptote 
Meet. 2O and the bias coefficients for the three degree (D) 
regression are: as G50 . a0 = =O s2 le 8 3 = 3659.04. 

The variance regression coefficients are determined from 
the variances of the population of estimates for each sub- 
sample size (dependent variable) and evaluating the single 


fjmecression 


Za 





+ + 


A Z il 1 
VAR (6 (nj) ) aie + o., apap ecb oe ai TR 








Memte(n,)) =o, 1 +o, 1 +... To = 
Ny ni- (d+2)/2 
k n 
k 
ieee yr 
= SS 3 A | Z 
where VAR (6 (n;)) = m=l i=l [9 () moi My) 
TM - J 
k=l, ak 
Mok 
6; (m,) 
aw = cra Sal 
d . — m=l i=l m-1‘"k 
an 5 (nz) k=1, x 
TM 


The four (D+l) variance regression coefficients for Figure 4a 
genet, = 08.57, a> = 174.72, ag = -1500.13, and a, = 5446.25. 
The variance regression coefficients derived here can be used 
memobtain an approximate variance for a specified subsample 
size by substituting the subsample size for n, in any one 
of the above regression equations along with the variance 
coefficients C4 Qo y ...)} and completing the computation. 
Below the variance regression coefficients the minimum 
and maximum value of the vertical scale is printed. The 
purpose of this information is to allow the user to scale 
Other estimators of the same parameter to the same scale. 


It can be used when the number of estimators exceeds three 





(the maximum number of estimators RAGE can use in a single 
Bum) Or when subsequent runs for different conditions are 
desired to be of the same scale for graphical comparisons. 

A final identifying feature of the graphics is a section 
at the very bottom of the graphical output which allows the 
user to identify the estimator that was used in estimating 
Mies parameter. in Figure 4a this estimator was the moment 
Sweamator, i.e. the reciprocal of the estimated coefficient 


Seevariation squared. 





ir. eS Chmse LLONGOr RAGE ARGUMENTS 


The RAGE program has the calling sequence: call 
Pe Ney, DyRhG, Obl, ovo, YMIN, YMAX,NEST ,EST1,TIL1,ESTZ, 
ieee, bolS,1TTL3). The arguments are relatively easy to 
understand but in several cases are related and must meet 
Memetain restrictions. For this reason a detailed descrip- 
tion of each argument 1S given, as well as any restrictions 
or limitations that must be met within or between arguments. 
Following this detailed discussion an example is given in 
Chapter IV on how to set up the main program and the estima- 


meeetunctions. 


A. DATA ARRAY...X 

The X argument is a single precision Real*4 array con- 
taining the data from which the parameter is to be estimated. 
ime only restriction is that the size of the X array must 
not exceed 50,000 which is the maximum sample size that RAGE 


@am Store. 


Pee OAMPLE SIZE...N 

The number of data elements per section or sample size 
is designated by the integer N. Depending on the minimum 
sample size required by the estimator, N can vary from 1 to 
20,000. The sample size, N, times the number of sections, 


M, cannot exceed 50,000 (the maximum number of data elements 
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stored in the array X). If M times N exceeds 50,000 an 

error message will be printed and execution will terminate. 
If M times N exceeds the number of data elements stored by 
the user in the array X then no error message will be printed 
and the program will execute giving unpredictable results or 


an abnormal ending will occur. 


fee NUMBER OF REPLICATIONS...M 

iie Munmben Of Sections Or replications the array X 1s 
sectioned into is determined by the integer value assigned to 
M. The number assigned to M is also the number of regressions 
that will be run to determine the regression coefficients 
ie e1> --8g- the number of regressions is limited to one 
hundred and therefore M is limited to a maximum value of one 
hundred. An error message will be printed and execution will 
terminate if M is greater than one hundred or less than one. 
If M is equal to one the variance and standard deviation of 
the regression coefficients for the unbiased estimate will 


not be printed since they will not exist. 


Meee SUBSAMPLE SIZE ARRAY...NE 

The argument NE is an integer array of size eight con- 
taining the subsample sizes (Ny k=1, ..., K) the estimator 
is to use in estimating the parameter. The values in the 
array NE should be ordered from smallest to largest or a 
warning message will be issued to indicate the vertical 


scale of the graph may not be large enough to include all 
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estimates. RAGE will use the population of estimates deter- 
mined from the subsample size stored in NE(1) to determine 

the vertical scale of the graph. If NE(1) is not the smallest 
value the range of the graphics may not be large enough to 
include outliers determined from subsequent smaller subsample 
sizes. 

Due to storage limitations a maximum of 12,500 estimates 
mepermissable. Since NE(1)}) will produce the most estimates, 
the user can determine the total number of estimates using the 
formula Mx[N/NE(1)]. If this value exceeds 12,500 an error 
message will result stating this limitation has been exceeded 
and execution will terminate. Corrective action can be taken 


by increasing NE(1) or decreasing M or N. 


meeeUMBER OF SUBSAMPLE SIZES (BOX PLOTS)...L 

ihe user can select the number of subsample sizes to 
be used from the array NE(8) by assigning an integer value 
to the argument L. The argument, L, can take on values 
between one and eight. RAGE will then select the first L 
subsample sizes from NE(8), compute the population of esti- 
mates of the parameter with each, graph the box plots and 
mmc the associated statistics. If L is not an integer 
value between one and eight an error message will result and 
EeeecuciOn will terminate. 

If L = 1 no regressions are possible and any regression 


Output is omitted. The maximum degree of regression allowed 
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by RAGE, regardless of the value assigned to the argument D 
(degree of regression), is L-l. If D exceeds this value RAGE 


will set the degree of regression to L-l. 


meee GkeEE OF REGRESSION. ..D 

The degree of regression, D, is an integer value less 
than or equal to three. The degree of regression refers to 
the number of bias coefficients, By> Bo, --8q, that will be 
used in computing the unbiased estimate out to D terms. 

In addition to the restrictions on D mentioned earlier, 
D will also be reduced for the regression that determines 
the variance coefficients, a, oy + Cay ds when the number of 


subsamples sizes with a variance is less than D+l. 


fee REDUCED GRAPHICS...RG 

The reduced graphics argument, RG, allows the user to 
reduce the range of the vertical scale of the graph by elimi- 
Meine outliers and only printing the number of outliers 
jeer exist. the vertical scale is reduced to the upper quar- 
me plus 1.5 times the interquartile distance value and the 
meer Quartile minus 1.5 times the interquartile distance 
value of the estimates from the smallest subsample size, 
NE(1). The vertical scale then becomes these values and the 
Memeeters are merely counted and printed on the graph at the 
Sees Of each box plot’s upper (lower) quartile + (-) 1.5 
times interquartile distance position. If there are no out- 


liers for the NE(1) estimates, either or both extremes, the 
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mertical scale is further reduced to the first estimate 


Value within these limits. RG = 1 will cause reduced graphics 


to be produced and RG = 0 will cause full scale graphics. 


fee SCALING ESTIMATORS INDIVIDUALLY...SEI 

The argument SEI allows the user to scale each estimator's 
femttical scale for the graphics separately or to the same 
eae. oLL = 0 camses all estimators’ vertical scale to have 
the same range of values. The range of the vertical scale 
Pmeewnether outliers are printed or counted will depend on the 
graphics desired (RG = 0 or 1). Figures 4a, 4b and 4c are the 
results of three estimators scaled to the same scale (SEI = 0) 
meee tull scale graphics (RG = 0). Figures 5a, 5b and 5c are 
the results of three estimators scaled to the same scale 
meet = 0) but with reduced graphics (RG = 1). 

If SEI = 1 each estimator's vertical scale will be scaled 
m@evidually. The range of the vertical scale and whether 
meirers are printed or counted will depend on the graphics 


desired (RG = 0 or 1). 


Meeecoe it iNG THE VERTICAL SCALE...SVS 

To set the vertical scale an integer value of 1 is assigned 
Memene argument SVS. SVS = 1 will cause the SEI argument to 
be ignored and the vertical scale of all the estimators' 
graphics will be set to those values assigned to YMIN and 
meee When the argument SVS is in effect (SVS = 1) the 


argument RG is also in effect as far as whether outliers 
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are printed or counted (RG = 0 or 1) but the determination 
of the vertical scale by RG is overridden by SVS and is set 
to YMIN and YMAX. SVS = 0 causes RAGE to ignore values 
assigned to YMIN and YMAX and scaling is determined by the 
values assigned to the arguments RG and SEI. Figure 4d is 
an example of SVS = 1 and RG = O with the vertical scale 
set to that of Figures 4a, 4b, and 4c (YMIN=-1.2812, 
YMAX=13.9984). Figure 5d is an example of SVS = 1 and 

RG = 1 with the vertical scale set to that of Figures 5a, 


5b and 5c (YMIN=1.0394, YMAX=9.1423). 


J. MINIMUM VALUE OF THE VERTICAL SCALE...YMIN 

The value YMIN is a real number assigned by the user as 
the lower limit of the vertical scale. If YMIN is assigned 
a value that is so large that it falls within the range of 
values that determine the body of any box plot then an error 


will occur and an abnormal ending will result. 


PeeetAXTMUM VALUE OF THE VERTICAL SCALE...YMAX 

The value YMAX is a real number assigned by the user as 
Miesupper limit of the vertical scale. If YMAX is assigned 
a value that is so small that it falls within the range of 
values that determine the body of any box plot then an error 


will occur and an abnormal ending will result. 


Eb. NUMBER OF ESTIMATORS...NEST 
jme argument NEST is an integer, 1, 2 or 3, which indicates 
mes mnumber Of estimators that will be used to estimate the 
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Megane ter. Al e@rrer message will be issued if NEST is not 


ome of these values and execution will terminate. 


Perot OR PUNCTIONS...EST1L,ESTZ,EST3 

These arguments are the function names assigned to each 
of the estimators that are to be used in estimating the para- 
meter. Each function must be written with a two argument 
calling sequence (i.e. call EST1(X,N)) where the first 
argument, X, must be the data array and the second argument, 
N, must be the number of data elements. 

If there are less than three estimators then dummy argu- 
ments must be inserted in the calling sequence where subse- 
quent estimators would normally appear (Suggestion...use a 
Punction of a previous estimator, i.e. EST1, as the dummy. 
mot Will not provide duplicate output for these repeated 
memetions unless NEST is increased to include them.). Each 
function must then be declared external in the main program. 
For an illustration of how the functions are written and 


declared see the example in Chapter IV. 


See oCRIPTION OF ESTIMATORS...TTL1,TTL2,TTL3 

The titles associated with each estimator can be passed 
@eeen argument through the call program (RAGE) provided 
there is a minimum of 120 characters between apostrophes 
ueeercall RAGE(,.., EST1,'---120 characters----',ESTZ,...)). 
An alternate and possibly easier method is to declare each 


Meee ({TTL1, TTLZ, TTL3) as a Real*8(15) array and assign 
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the characters that describe each estimator as data to each 
title. For an example of this method see the example in 
eaapter IV. 

The user should note that the title associated with each 
estimator must immediately follow the estimator argument. 
As in the case of estimators, dummy titles must be inserted 
when the number of estimators requiring titles is less than 
@ence. Finally, it should be noted that the program will 
not execute properly unless titles of the correct format are 


fmoeluded. 





IV. EXAMPLE OF A PROGRAM USING RAGE 


In the following example suppose a user has a set of iid 
data from which he wishes to estimate the population mean by 
Meng two estimators, the sample mean and the sample median. 
Assume that these estimators are given the abbreviated func- 
t10On names MN and MED and that the associated titles for 
each estimator are "''The sample mean as an estimator of the 
population mean'' and "The sample median as an estimator of 
the population mean."' If the subsample sizes at which it is 
Moatred tO €Xamine the properties of these estimators are 
wou, (1, 125, 166, 200, 250 and 500 the FORTRAN program 


may look as formatted on the following page. 





REAL*4 X(50000) ,YMIN, YMAX 

REAL*8 T1(15),T2(15) 

INTEGER NEST,D,RG,SEI,SVS,M,N,L,NE(8) 

DATA NE/ 33,50,71,125,166,200,250 500/ 

INSERT 8 CHARACTERS BETWEEN EACH SET OF APOSTROPHES UNTIL 
TITLE IS COMPLETED AND FILL REMAINDER WITH BLANKS. 

DATA T1/'THE SAMP','LE MEAN ','AS AN ES','TIMATOR ', 
+'OF THE P','OPULATIO','N MEAN',8*! '/ 

DATA T2/'THE SAMP','LE MEDIA','N AS AN ','ESTIMATO', 
+'R OF THE','POPULATI','ON MEAN',8*! '/ 


EXTERNAL MN ,MED 


CALL RAGE(X,N,M,NE,L,D,RG,SEI,SVS,YMIN,YMAX, 


Pibot sin Ti, MED, 12 9iN,T1) 


SOP 


END 
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REAL FUNCTION MN(X,N) 
REAL X(N) 
AVERAGE OF N VALUES IN X 
MN=0.0 
1 LG tien ay 

MN = MN +X(T) 
CONTINUE 
MN = MN/FLOAT(N) 
RETURN 


END 


REAL FUNCTION MED(X,N) 

REAL X(N) 

MEDIAN OF N VALUES IN X 

"PCTILE" IS A FUNCTION THAT RETURNS THE 50 PERCENTILE 
NUMBER FROM THE DATA ARRAY X OF SIZE N. 

MED = PCTILE(X,N,.5) 

RETURN 


END 


Im this example since only two estimators were used a dummy 


estimator and title had to be inserted as the arguments for 


the third case. Here the first estimator and its title were 


Simply repeated. 





V. A COMPARATIVE STUDY OF ESTIMATORS FOR THE GAMMA SHAPE 


PARAMETER 


To demonstrate the use of the Regression Adjusted Graphics 
and Estimating (RAGE) package, two arrays of gamma distri- 
buted random variables with shape K=5.0 and K=0.2 were 
generated using the Naval Postgraduate School's random number 
generator--LLRANDOMII [Ref. 3]. Because these random numbers 
are independent and were drawn from the same distribution, 
they can be sectioned into independent and identically 
distributed blocks of size N. RAGE was specifically designed 
to evaluate estimators of parameters from just such iid 
populations. These two samples then should demonstrate the 
Capabilities of the RAGE package to evaluate the bias, vari- 
ance and distribution of estimators of a specified parameter 
from the populations. 

in the following examples it is the shape parameter, K, 
of the gamma distribution that is being estimated. There are 
two primary estimators used in these examples. The first is 
the moment estimator (reciprocal of the estimated coefficient 
of variation squared), K = _ = =e The second 

(C(X)}* — VAR(X) 
estimator is the maximum likelihood estimate (MLE) [Refs. 2,4] 


which is determined by a bisection search for the shape para- 


Meter, K, until the following equation is satisfied: 


So 





n 
Slim 
i=] 


nN 


Ln(k) - Digamma(K) = Ln X - 


The Xa 1=1, 2, ..., n are the random numbers from a specified 
block of size n. The other two estimators investigated are 
jacknifed versions of the initial estimators. The jacknife 
routine [{[Ref. 5] with each of these estimators is a four fold 


jacknife which has the form 
a0 ul = % 
4 j=l 


where 84; = ACen) = 3 (64), Gail 1s the estimate from the 
eier1ce sample (n) and 8; 1s the estimate with the ioe group 
missing. Each group is a set of one fourth of the data 
moants. 

Each of the four estimators is examined for varying sub- 
sample sizes and replications to determine how accurately 
fea eStimator estimates the shape parameter of the gamma 
distribution as the subsample size increases. The shape 
parameter of the gamma distribution was chosen as an example 
because the behavior of the many estimators which have been 
proposed for K is fairly well known. Roughly speaking, it is 
known that the moment estimator is quite efficient for K>2 
but quite inefficient for K<l even for large sample sizes. 


Bias and normality of these estimators is not documented. 


Again the jacknife is known to work well with moment estimators 
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even in small samples but no such results are known for the 
jacknifed MLE estimator except that it is asymptotically 
eel cient. 

For this investigation there are basically three examples 
Mepresenting different simulation scenarios for evaluating 
each of two cases, K=5.0 and K=0.2. They are: 

(a) Small subsample sizes (33 to 500), small 

replications (M=2); 

(b) Small subsample sizes (33 to 500), large 

replications (M=100); 

(c) Large subsample sizes (100 to 2000), large 

replications (M=25). 
ire tirst can be looked on as a pilot study to determine 
Meo1ons of interest for the two longer scale studies. How- 
Meemeetine procedures are not fully sequential, unlike the 
usage of the RAGE methodology proposed by Heidelberger and 
Lewis {Ref. 1] for regenerative simulations. 

[eememamye ot these examples another set of reduced graphics 
mememeluded which reduces the vertical scale and provides 
graphics which are more suitable for examining the bias 
feereture of the estimators. The reduced graphics truncates 
meemoutiiers in the box plots giving only their number and 


> 


Mmoeee their position. 


A. EXAMINING THE GAMMA DISTRIBUTION FOR THE SHAPE PARAMETER 
(K=5.0) 


1. Case 1, Example 1: Small Subsample Sizes and 

Puaures sa, SD yeoemand Sdeshow the graphical. and 
Seaeistical results from estimating the shape parameter with 
the four different estimators. These graphs all have the 
Same vertical scale so that meaningful comparisons can be 
made of the box plots and of the bias of each estimator. 

CiewemmrNeeGiaracteristics of the MLE estimator is 
that its estimates will asymptotically have the smallest vari- 
ance of all estimators. For small subsample sizes (such as 
the case of Figures 3a § 3b) the asymptotically smaller vari- 
ance of the MLE estimator is not apparent in the box plots. 
mecmpody Of the box plot for the MLE estimator for nj = 33 
1s smaller than the same box plot for the moment estimator. 
However, the MLE has more and larger outliers, indicating 
that the MLE is more nonnormal at this sample size than the 
moment estimator. If, however, the estimated standard devia- 
|™10n of the various subsample sizes are compared for each of 
the two primary estimators (Table 1) it can be seen that the 
fimenestimator consistently produces a smaller variance than 
does the moment estimator. From Figures 3a §& 3b the bias in 
the two estimators appears comparable, starting high but 


becoming negligible by subsample size 250. 





Table 1: Standard deviations of estimates of the shape 
parameter using the moment and MLE estimators 
for small sample sizes (replications = 2) 
Standard Deviation 


SAM EBs tle MOMENT ESTIMATOR MLE ESTIMATOR 


bys) one, Heres) US 
20 wo oAZ Sells le 
#1 HOI a 9) jon 
100 Oro lee ~4612 
Ws SH GAZ ~4823 
166 7S om0 sll4 
250 -4660 MY acc 
5900 oi, BS (tom 


Jacknifing is a technique that can be used on iid sam- 
ples to reduce the bias in the estimates caused by the estimator 
[Ref. 5]. An estimator with a complete jacknife is asymptoti- 
cally normal with the same variance as the unjacknifed estimator, 
but small sample properties are difficult to obtain analyti- 
cally. Figures 3c & 3d are the jacknifed estimators and 
again because of the small number of estimates it is not 
Meearent that the jacknife 1s contributing to nonnormality 
Or increasing variance when the box plots are compared with 
those of Figures 3a & 3b. The increased normality, however, 
can be seen in the jacknifed moment estimator by comparing 
box plots (Figures 3a §& 3c) and the coefficients of skewness 
(Table 2) with the nonjacknifed form. The change in normality 


Or symmetry of the estimates from the jacknifed MLE estimator 
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is not discernable in a comparison of the box plots (Figures 


3b & 3d) or the coefficients of skewness (Table 3). 


Rabble wero kemmesseGCoctticr1ents Of the estimates of 
the shape parameter using the moment and 
jected Moment eStimators (replications = 2) 
Skewness Coefficients 


SaMPLE SIZE MOMENT ESTIMATOR JAC KN DRED rOKM 


5 Poo O27, 
50 .076 a SS 
tal 6 SS, Ay 
100 499 silos 
125 eS .724 
166 1.018 =o 011 
7540) 250 See 
500 N/A N/A 
Mawle 3: Skewness coefficients of the estimates of the 
Shape parameter using the MLE and jacknifed MLE 
estimators (replications = 2) 


Skewness Coefficients 


SAMPLE SIZE MLE ESTIMATOR JACKNIPED FORM 
ee. Ie db See 0150 
20 25180 oie 
fal »193 ore oe 
100 sees: 645 
125 eer Jl Se) 
166 ee Oa deo 
25.0 Seow = Oral 
500 N/A N/A 
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The bias in each estimator is depicted by the space 
between the dotted line and dashed line of each of the graphs. 
When the unbiased estimate (dotted line) approaches the 
estimated expected value (dashed line) then the bias becomes 
insignificant. Each estimator can be expected to vary in 
the amount of bias it has in its estimates and this bias can 
be expected to diminish as sample size increases. Comparing 
meeures Sa & 5b 1t appears that the MLE estimator has about 
the same bias as the moment estimator but this is hard to 
determine at this point since the bias coefficients were 
determined from a small number of estimates. In the esti- 
mates from the jacknifed estimators one would expect the bias 
to be reduced because jacknifing is specifically designed to 
memove the 8, {l/n) term in the bias of the estimates but 
this is not obvious from the graphical comparisons (Figures 
3a & 3c and Figures 3b & 3d). For the case of the MLE esti- 
mator the coefficient of the l1/n tern, Sig gels cle EVI, 
increased from 32.65 to 75.35 after jacknifing. The standard 
deviation of these bias terms are so large (19.14 and 16.23 
respectively), though, that they suggest more estimates 
Mancreased replications) are necessary to better determine 
the bias. 

Determining how much more bias one estimator has than 
another can also be accomplished by comparing the bias 
Meyession coefficients 8, , 84, etc. The larger these coef- 


ficients are the greater is the bias of that particular 
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Swimacor. Comparing the coefficients for the moment and 

MLE estimators of Figures 3a §& 3b show that the MLE estimator 
fas a larger bias. For a subsample size of 500 and a three 
degree regression the bias for the moment estimator is 


7.40128 a ils) Sy EOE, Fi BB 010 


= Ons 
500 5002 500° 
ema the bias for the MLE estimator is 
ANN 1998.68 47308.0 
eee ee SCdS77 
500 500- 500° 


These comparisons seem to indicate that the MLE estimator 

has more bias. This is not necessarily true, though, for 
reasons mentioned earlier. Increasing the number of replica- 
tions will alleviate much of this uncertainty since more 
precision in the estimates of the bias coefficients will 


result. 
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2. Case 1, Example 2: Small Subsample Sizes and Large 
Rep ireanrens (K=5.0) 


It was pointed out in Example 1 of this case that 
the small number of replications left some doubt as to the 
accuracy of the estimates of the bias coefficients as well 
as the normality of the estimates. Figures 4a to 4d (cor- 
responding to Figures 3a through 3d) are for increased 
replications (M=100). This has the effect of increasing the 
population of estimates for each subsample size (ny, | are eae 
K) and the number of estimates of the bias coefficients 
wee, ~-.-) oUt holds the subsample size at which the esti- 
mators are examined small as before. On first observing the 
graphs, it becomes apparent from the box plots of the two 
Suatlest subsample sizes (n,=35 and nj,=50) that the estimates 
are nonnormally distributed (Figures 4a §& 4b). The jacknifed 
estimators remove some of this nonnormality (Figures 4c §& 4c), 
especially for the jacknifed moment estimator. In effect, 
outliers appear now at both high and low values but consider- 
able skewness still remains. 

Hermenwlt iS apparent by the statistical output that the 
MLE estimator has significantly less variance in its popula- 


tion of estimates than does the moment estimator (Table 4). 
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Table 4: Standard deviations of the estimates of the 
shape parameter using the moment and MLE 
estimators for varying sample sizes 
(replications = 100) 


Smandanra Deviation 


SaviPbe O12 MOMENT ESTIMATOR MLE ESTIMATOR 


a 1.486 12528 

20 J agi) lag ges Ie bes 

fl Spode ye We, pOo4 9 
100 8008 POS 6 
Las e242 echoes hee: 
166 .6499 woa4 5 
2, S10, 55.08 2 -4036 
900 ~5417 we 0 


The increased symmetry and normality by the jacknifed Tower 
estimator is graphically visible if Figures 4a §& 4c are com- 
pared. The change in symmetry or normality of the estimates 
of the jacknifed MLE estimator was not discernable in the 
graphics (Figures 3b § 3d) or skewness coefficients (Table 3) 
for Example 1. However, for increased replications, (M=100) 
if the box plots of Figures 4b §& 4d are compared it appears 
that the population of MLE estimates are more normally 
distributed (outliers at both ends) after jacknifing. The 
movement toward symmetry is more obvious by comparing the 
Skewness coefficients of the MLE estimates and the jacknifed 


MLE estimates. 
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iicmvocranmecencauction, or lack of it, for each 
estimator when it is jacknifed can be determined by comparing 
the computed variances that result from a given subsample 
Size and the variance coefficients, G1, A, etc. lipase Preps 
Seample, the variance coefficients (a), Ao» AF, and ag) for 
each of the estimators (Figures 4a through 4d) are used. to 
determine the variance for a subsample size of 500, the vari- 


mmees tor each estimator would be: 














Se ry ae ee PHS, ORs 1273 
500 soot? 5002 5002"? 

VAR (MLE) Dees 2 961.91 | 10183.5 DE ae 
500 s00L-> 5002 500¢°° 

ONT me eee eet 357 
500 s00l-5 500° 5007-9 

Meee mEE)y= COS = Bene, BONO eee te os 10827 
500 s00t°? 500- 500°"? 


These computations show a lack of change in the variance 
when the estimators are jacknifed and agree with the statis- 
meedt and graphical conclusions drawn earlier. 

ies Drassin the MLE and moment estimators is difficult 
to compare graphically (Figures 4a §& 4b) since the unbiased 
estimate approaches the estimated expected value so quickly 
and because of the compacting caused by the range of the 
vertical scale. For this reason reduced graphics are produced 


(Figures Sa through 5d), which the reader will recall 
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Suppresses tne OUtliersS. Since the scale is reduced from a 
Homeeror 15.28 (14,00 = (-1.28)) to 8.10 (9.14 - 1.04) or 
almost 50%, the evolution of the bias as subsample size 
increases becomes clearer, as Figures 5a §& 5b demonstrate in 
comparison to Figures 4a §& 4b. These graphs indicate that, 
for at least small subsample sizes, the moment estimator is 
a more biased estimator than the MLE estimator. 

Reduced graphics are not of much use for the jacknifed 
estimators (Figures 5c §& 5d) since the bias is almost conm- 
pletely removed by jacknifing. They are useful only in the 
sense that they are on the same scale as their nonjacknifed 
Pais and a2 Comparison of bias between estimators can only 
Memaone graphically if the graphics of estimators are on the 
same scale. This, however, was already accomplished from 
Figures 4a through 4d. 

As was done for the variance, the amount of bias for 
eeciven subsample size can be estimated by dividing the coef- 
ficients, Sy» By, €tc., by the sample size squared, cubed, 
eee) aS appropriate and summing (See Chapter II “USAGE” for 
the mathematical formula). The computations will not be 
Carried out here as in the case of the variance but it is 
useful to compare the Sy SCC eteomus Oru bne CStimatonrs 
memeemtnese coefficients are likely to contribute the most 
Significant amount of bias (they are divided by the smallest 


denominator). In the case of the moment estimator, eos 
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mmeoledna tor the MLE e€stimator it is 14.77. This suspected 
Mmeeater Dias by the moment estimator can be seen visually 
meame tne graphics. The smaller coefficients for By GE tne 
jacknifed moment and MLE estimators are 1.47 and 4.09 respec- 
tively. This gives an indication of how much less the bias 
becomes when the estimators are jacknifed. This kind of 
comparison of By coefficients should be done with caution, 
though, since subsequent coefficients (85, Ba, Se ita y be) So 
large that they contribute more to the bias than the By coer - 
mmememt. It should also be pointed out that these By SGen ii = 
cients should be zero because, as mentioned earlier, the 
jacknife removes the 8, (1/n) term. In fact, their estimated 
Beeanaard deviations are 4.00 and 5.6/7 respectively, so that 
they are within one standard deviation of zero. 

PNP OmaiemsmEne size wor these bias CoOerricients 
meemeneir accuracy. Comparing the bias coefficients from 
Figures 5a through 5d in relation to their standard deviation 
Meee those Of Figures 3a through 3d (Table 4) which were 
@etermined from only two replications, shows the advantage 


Suebarce replication in estimating the bias of an estimator. 
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apices:  btas COerriTeclents fOr each estimator's 
population of estimates and their standard 
dew wicCtiweuotesiall and large replications 
(M=2 and M=100) 
BIAS NUMBER OF REPLICATIONS 
ESTIMATOR COEFF. TWO (2) ONE HUNDRED (100) 
84 7.40 16.49 
(13.5) (2m 2) 
MOMENT B 5 Ute i eal = 1b 5 
CIEE) Ses) (20415) 
Bz 1947.9 3659.04 
(21284.5) (4 5'2:6" 5 ) 
By S202 14. aw 
le Pee i (Zea ) 
MLE B 5 = WSS) aay Sew eZ 
(uci 2) (Z2NEey 
83 47308.0 LOO2 224 
no Oi Sek } (4798.3) 
54 6.59 Lea 
Gie20) (4.01) 
JNK MNT 35 =O) ey 9.75 
(TOmk6 24 ) Go Se-o0) 
8 3 PMT VE S69 220 
(e259 8e.6 ) (7411.7) 
54 aso 4.09 
(p62 2.53) (3.599) 
JNK MLE 55 obo. 0 =2 cee 
(1450.6) (436802 ) 
8 3 ls) 2-0) eo Seales: 
(5205 eas) (9245202) 
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Figure 5d (REDUCED GRAPHICS) 
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eecser le oxample 3S: Large Subsample Sizes and Large 
Replications (K=5.0) 


Figures 4a through 4d showed the necessity of increas- 
Migeecupsample Sizes in Order to examine the normality of the 
estimates. Figures 6a through 6d are for increased subsample 
sizes (100 to 2000) with a large number of replications (M=25). 
The box plots in these figures show the increase in normality 


With increasing subsample size n, when they are compared with 


k 
small subsample sizes (Figures 4a through 4d). The reduction 
tievanrtability is also obvious by noticing the lack of outliers 
in the box plots and the smaller range of the vertical scale 
Goebarger sample sizes. The standard deviation for each of 
the estimators for small subsample sizes (Nng=500, Figures 4a 
through 4d) is compared to those for large subsample sizes 
(n,=2000, pares 6a through 6d), Table 6. It can be seen 

that the variance is reduced for larger subsample sizes by 

the appropriate factor of two. However, the degrees of freedom 
in the standard deviation of the estimators are 25 at Ne = 2000 
ama) 100 at ng = 500. Again it is clear from these figures 

that jacknifing causes no lack of precision in either the MLE 
Smemoment estimators. 

The bias in the moment and MLE estimators is difficult 
to compare from the graphics because of the range of the verti- 
Mmieeeocale. For this reason reduced graphics (Figures 7a §& 7b) 
for these two estimators are produced. Reduced graphics for 


the jacknifed estimators would not be very useful since it 


ope 





has been shown in previous examples that very little bias 
meeests., For this reason they are omitted. 
iitele G: Standard deviations of the population of 
SScimaeesOr tile’ Shape parameter for each 


estimator for subsample sizes 500 and 2000 


Se eo Leo 


ESTIMATOR Ng = 500 Mae = Guu 
MOMENT . 5417 woo 2 
MLE Ba 20 SUS IIUG 
JACKNIFED 
MOMENT - 5400 Sie 
JACKNIFED 
MLE ee, oOo 0 


A change seems to have occurred if a comparison is 
Mere or Figures 7a G& 7b with those of Figures 5a & Sb. The 
graphics seem to indicate that the MLE's unbiased estimates 
have shifted from being biased on the high side to the low 
Side for subsample size n,=100 and possibly n,=200. Also, 
it appears that the two estimators have about the same bias 
Or that possibly the moment estimator has less bias. Looking 
PUrther at the bias coefficients that determine these graphics, 
tt Can be seen that the coefficients in Figures 5a and 5b are 
Poeimated with more precision than those of Figures 7a and 
7b. One may be tempted to assume that the MLE estimator is 
MemesbDiased for larger sample sizes. But this is not likely 


and the change in the bias structure for Figure 7b can be 


62 





attributed to the difference in the values of the bias 
coefficients (85, 85; ..) compared to those of Figure 5b. 

These coefficients should be the same but the coefficients 

of Figure 5b were computed from 100 estimates (100 regressions) 
and those from Figure 7b only 25. The precision can be 
eeeected to be better in the first case. This is obviously 

the case if the standard deviation of the coefficients are 
compared, and more confidence can be placed in the bias 


structure of Figure 5b. 
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B. EXAMINING THE GAMMA DISTRIBUTION FOR THE SHAPE PARAMETER 
(K=0.2) 


1. Case 2, Example 1: Small Subsample Sizes and 

Sid eoercd ELOn's (K=0.2) 

In the preceding case for K=5.0 four estimators of 
the shape parameter of the gamma distribution were evaluated 
for varying subsample sizes and replications. This evaluation 
1s repeated here with the shape parameter changed from K=5.0 
Memnr-0-2. This second set of comparisons is not only useful 
in demonstrating how the RAGE package shows the behavior of 
different estimators in regards to bias and variance, but 
also how these estimators may differ when the parameter being 
evaluated takes on different values. 

Figures 8a through 8d are graphical results of the 
four estimators of the shape parameter, K=0.2. For these 
small subsample sizes and small population of estimates it 
immanparent from the graphics that the MLE estimator (Figure 
8b) has tighter box plots than does the moment estimator 
(Figure 8a). This shows graphically that the variance for 
the MLE estimator is less than that of the moment estimator. 
This was not obvious when the shape parameter was 5.0. In 
ieeeuices Sa & Sb of Case 1, Example 1, it was difficult to 
distinguish any difference in the box plots. This comparison 
shows how the MLE estimator is much better at reducing the 
variance when the actual shape parameter is as small as 0.2. 


Or, alternatively, it may be an indication of how much worse 
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the moment estimator is for estimating shape parameters 
that are small. 

Figures 8c §& 8d are the jacknifed forms of the esti- 
mators. It is not at all clear from the graphs that the 
jacknife is affecting the variance of the estimates. This 
lack of change could be attributed to the small number of 
estimates which suggest that the number of replications 
should be increased to achieve a better graphical evaluation. 

In Case 1, Examples 1, 2 and 3, the coefficients of 
variance (a4, Ao, Os and a4) were helpful in determining 
which estimators produced the least variance. A look at a, 
in Figures 8b and 8d shows that they are negative numbers. 
Since aby often contributes the most significant amount to 
the variance, there is some suspicion that a determination 
Seeene variance by the use of these coefficients could result 
in negative variances for the MLE estimator and its jacknifed 
fonum. If 2., and a, are not large positive numbers (relatively) 
Meets especially likely. In these two cases a7 is a rela- 
tively large positive number and negative variances do not 
occur if the calculations are carried through. However, 
for lesser degrees of regression (i.e. D=2) negative variances 
eam occur. f[fhis is a result of the wide fluctuation of the 
variances of the small subsample sizes whose values are used 


as dependent variables in the regression which determine 


Ap» ty; and Os. These variance coefficients are often 


Til 





questionable when the variances of the population of esti- 
mates vary over a wide range and further suggest increasing 
the number of replications so that the variance coefficients 
can be more accurately determined. 

The bias in the moment estimator (Figure 8a) appears 
to be considerably more than that of the MLE estimator 
(Figure 8b) when the graphs of figures are compared. This 
Dias is further reduced when the jacknife is applied to the 
estimators (Figures 8c 4 8d). A comparison of the bias is 
inadvisable at this point since the bias coefficients have 
Barge Standard deviation values in comparison to their 
estimates. The By coefficient, for example, of the MLE 
estimator is .0864 and its standard deviation is .189. As 
mentioned in previous examples, 2, will normally contribute 
the most to the bias and since its standard deviation is 
larger than the estimate itself, its accuracy is highly 
questionable. Before any firm conclusions can be drawn 
Semcerning the bias of the estimators more estimates are 


required, which implies increasing the number of replications. 
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2. Case 2, Example 2: Small Subsample Sizes and 
Large Hep. 1ceat 2ons (K=0.2) 


Patines decir OUgm IJdeare tor small subsample sizes 
but large replications (M=100). This increased number of 
Pewimates produces more definitive graphics for better 
evaluations of the bias and variance of the estimates pro- 
duced by the estimators. 

them eraphis Ot ene emomenc and MLE estimators (Fipures 
9a & 9b) show clearly as before that the MLE estimator has 
much less variance than does the moment estimator. Both, 
however, have positive skewness and high nonnormal estimates 
up to subsample size n,=125. emiatedtietec™ Ot Ot stine 
moment estimator has about the same variance as its nonjack- 
nifed form but is more symmetrically distributed. This can 
be seen in a comparison of the box plots of Figures 9a §& 9c. 

The distribution of the population of estimates for 
the jacknifed MLE estimator shows little if any difference 
Seapatcally or statistically than the nonjacknifed estimates. 
This can be seen by comparing Figures 9b §& 9d, which show 
Memes Giftterence when the box plots are compared for 
Maeious subsample sizes. For large values of the shape 
parameter, i.e. K=5.0, it could be seen from Figures 4b § 4d 
fueease |, Example 1 that the jacknifed MLE estimator 
increased the symmetry of the estimates. This was obvious 
both in the graphics and by comparing the skewness coeffi- 


G@t@ents. For K=0.2 the only indication that the estimates 
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are more symmetrically distributed is the comparison of the 
skewness coefficients for small subsample sizes (n,=33, 

nz=50 and nee! 1) of the MLE estimator and its jacknifed form 
(Figures 9b & 9d). The skewness appears to decrease slightly, 
but not discernably, for larger subsample sizes. 

The amount of variance that was observed graphically 
can also be seen statistically by computing the variance of 
Saemeestimator for designated subsample sizes using the vari- 
ance coefficients Ap» 4, Ae and o 4 of each estimator. These 
@eetricients for each estimator will result in positive 
Variances. This is not always the case when the coefficients 
are determined from subsample sizes with a small population 
Smestimates. This example illustrates the increased confi- 
meiee that Can Dé put into the variance coefficients as the 
humber of estimates (replications) increases. 

The bias again appears to be substantially greater 
fOr the moment estimator than the MLE estimator for comparable 
subsample sizes. Here, because of the range of the vertical 
scale, reduced graphics are produced for the moment and MLE 
Eeeimators to better compare the bias of these estimators 
by reducing the scale (Figures 10a and 10b). <A comparison 
of Figures 10a and 10b clearly shows the increased bias 
caused by the moment estimator. It was shown in Example 1 
of Case 1 that the jacknifed estimators have little bias 


miaming In the estimates. This appears to be the case for 
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K=0.2, as indicated by the small bias coefficients of the 
jacknifed estimators (Figures 9c & 9d). For this reason, 
reduced graphics for the jacknifed estimators are omitted, 
Since indications are they would show little more graphically 
Maan Figures 9c qG 9d. 

{[t 15 interesting to note that the bias of the moment 
estimator compared to that of the MLE estimator is not only 
Beeater, but greater for small values of the shape parameter 
Mmeec. K=0.2) than for large values (i.e. K=5.0). This can 
Beeseen by comparing the bias in Figures 10a & 10b with that 
Seeerigures 5a 4 5b. This indicates that the moment estimator 
meenot Only more biased than the MLE estimator but consider- 
ably more biased when the shape parameter being estimated is 


small. 
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FOUR FOLD JACKNIFED MOMENT ESTIMATOR OF THE SHAPE PARAMETER OF THE GAMMA DISTRIBUTION. 


REGRESSION ON VARIANCE 
VERTICAL SCALES 
ESTIMATOR: 
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3. Case 2, Example 3: Large Subsample Sizes and 

As a last example, graphs for large subsample sizes 
with a large number of replications (M=25) are produced for 
each estimator. The box plots of these graphs (Figures lla 
mamough lid) look similar to their counterparts for small 
mamole Sizes and large replications (Figures 9a through 9d). 
This is due to the difference in the vertical scale and does 
not indicate a lack of variance reduction by larger sample 
Bazes. The range of the vertical scale of Figures lla 
through 1lld is .495 and is smaller than that of Figures 9a 
Memeo Sd, which is .9865, by a factor or two. This is a 
Simple but important illustration of the caution that must 
be exercised when comparing graphics that are not of the 
meme Scale. 

Other than the reduced variance caused by larger 
Sample sizes, the same conclusions that were reached in 
Example 2 of this case concerning the variance and the bias 
of the estimators is shown to hold here. The MLE estimator 
shows considerably less variance and bias than the moment 
estimator. The jacknifed moment estimator results in more 
normally distributed estimates and less bias. The jacknifed 
MLE estimator shows some improvement toward normality of its 
estimates and its variance has changed little. 

As in Case 1 for K=5.0, the larger the subsample 


Size the more normal are the population of estimates. This 
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increased normality can be seen graphically by comparing the 
graphs of Figures lla through lld with those of Figures 9a 
mmeouch 9d. the lack of outliers on the high side of the 

Mex plots for Figures 9a to 9d is an indication of convergence 
normality. 

hike we weslilndidaes tne nesults of Ehe Comparisons of 
the four different estimators for K=5.0 and K=0.2. A com- 
parison of the standard deviations shows the moment estimator 
to be consistently higher than the MLE estimator. These 
Seadcard deviations, however, show little change when the 
primary estimators are jacknifed, indicating little or no 
change in variance by jacknifing. The bias values of the 
moment estimator are consistently higher and the root mean 
Square error (r.m.s.e.}) is considerably larger for K=0.2 in 
comparison to the MLE estimator, indicating how poor the 
moment estimator is for evaluating shape parameters of gamma 
distribution that have small values. 

These conclusions were easily drawn from the graphs 
and the statistical output provided by the RAGE program and 
did not require extensive programming and analysis by the 
experimenter. This kind of expedient examination of estima- 
tors not only saves the experimenter time but reduces signifi- 
cantly the computer time required for analysis. This 
facilitation of analysis is the primary purpose of the RAGE 


methodology. 
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